Search API Design Decisions

Learn the workflow of a search API and important technical considerations that direct its design.

For a search API to work efficiently, there are a number of services operating at the backend. In the previous lesson, we learned a pivotal service (the search service) of the search API. A typical search API might utilize a few auxiliary services. We’ll consider some of them in this lesson. We’ll also look at some technical considerations that will help us to design an effective search API.

Design overview#

Consider the following diagram to see how the search API interacts with a number of services when a search query is received.

The end-to-end architecture of the search API
The end-to-end architecture of the search API

Let's discuss the role of each component and service shown in the diagram above.

Components and Services Details

Component or Service

Details

Search service

  • Performs searching, pagination, and sorting

Recommendation service

  • Recommends other search phrases related to the user's searched query


User service

  • Stores and provides user information to other services
  • Records the previous search history of users
  • Collaborates with the ads service for personalized ads
  • Interacts with the search service to filter or sort search results

Ads service

  • Returns advertisements based on the search query


API gateway

  • Authenticates and authorizes incoming requests
  • Throttles requests based on rate limiting
  • Caches the response to frequently made search queries
  • Routes the request to appropriate search, recommendation, and ads services

Database

  • NoSQL distributed database cluster to store the application data and index table

Note:  An alternative to the recommendation service can be the typeahead service but we avoided that functionality because it will increase the complexity of our design. Typeahead is a separate design problem, and we leave it to the learner for further exploration.

The following section discusses how each component and service in the search system interacts with each other to fulfill the client's request.

Workflow#

Since the main functionality of the API is to return search results, it’s safe to assume that most of the requests center around search capabilities. All the other services shown above interact with each other to exchange data relevant to the search query.

A typical search query will simultaneously reach all the services that will produce query-specific responses. While the ads and recommendation services will provide their respective responses, the search service will generate results in line with the information provided by the user service, and it will perform sorting and pagination based on the query parameters of the user.

The consolidated response is presented to the user via the API gateway. It should be noted that each query is stored in the persistence layer, which makes it available to all the services. This is done in order to generate intelligent results in the future.

Note: The server returns the data (title, description, and link to thumbnails) to the clients. The client may download the thumbnails from its nearest CDNs.

So far, we’ve looked at the overall workflow and the components involved in a search API. Let's now decide on some design considerations for the API.

Design considerations#

In this section, we describe various technical aspects that make our search API efficient. These aspects include architecture styles, data formats, and HTTP versions.

Architectural style#

Let's decide on the API architecture styles for the interaction between users, API gateway, and back-end services.

Client to API gateway#

In the case of a search service, a user only sends a query instead of creating or updating any resources on the backend; therefore, it can be easily performed via the GET HTTP method. So the obvious choice that can benefit us the most is to opt for REST between the client and API gateway.

The API architecture style between client and API gateway
The API architecture style between client and API gateway

API gateway to back-end services#

The operations performed are mainly resource oriented, and the API gateway possibly has to fetch data from multiple services. Therefore, with a quick look, gRPC is excluded because it’s action oriented and is better for intra-service communication, such as performing function calls very quickly and efficiently.

Now, we’re left with the REST and GraphQL options to decide on. We have a limited number of services to interact with, and services like recommendations and ads provide independent responses to the gateway. The REST architecture is suitable because the search service returns complete documents rather than custom fields and sections from different documents. The API gateway, after combining all results, provides the results to the client. On the other hand, GraphQL brings maintenance overhead and tightly couples the backend to the API gateway because the client must know the schema before retrieving data, and the clients must also update their implementation accordingly if we change the internal schema.

The architecture styles used between API gateway and backend services
The architecture styles used between API gateway and backend services

Data formats#

We employ JSON data formats for the communication between the interacting entities: client, API gateway, and back-end services. Query requests via the search API are made via the GET HTTP method, which contains no body; however, the data received should be readable to humans for debugging purposes and easily renderable by the browsers. The data in the response contains the searched and filtered data, including media files; therefore, a suitable option for REST is to go with the JSON data format.

HTTP version#

We select HTTP/1.1 between the client and API gateway because pagination requires a persistent connection to avoid remaking connections between the client and server. HTTP 1.1 added support for persistent connections, so we need that version. Moreover, we don't need multiplexing or streaming; therefore, we should avoid HTTP/2.0, which incurs other extra complexities.

Summary#

In this lesson, we discussed which architecture is suitable for designing our search API and explored the different features of search API. The following table summarizes the design decisions that have been taken in this lesson.

Design Considerations

Client to API Gateway

API Gateway to Backend

Architecture style

REST

REST

Data format

JSON

JSON

HTTP version

HTTP/1.1

HTTP/1.1

In the next lesson, we’ll define an endpoint for the search API and discuss the complete request and response structure.

Q

Is it possible to adopt two different HTTP versions—for example, to use HTTP/1.1 between the client and API gateway and HTTP/2.0 between the API gateway and back-end services?

Your Answer
A)

Yes

Explanation

Yes, we can perform the translation the same as the envoy proxy does. We consider the API gateway as an envoy proxy that will translate the request from one version to another version. There is a one-time creation cost for version translation because all the requests have to pass from the API gateway.

B)

No

Explanation

We cannot adopt two different HTTP versions without a proxy.

Introduction to the Search Service

API Model for Search Service